Virtual reality environment

ABSTRACT

A three-dimensional virtual reality environment.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional PatentApplication No. 62/654,488, filed Apr. 8, 2018, entitled Virtual RealityEnvironment.

BACKGROUND OF THE INVENTION

Referring to FIG. 1, a computer 22 often includes a microprocessor, amemory, a storage device (e.g., hard drive or solid state memory), agraphics processor that is coupled to an external display 18, such as aliquid crystal display. The computer 22 often includes input devicessuch as a mouse 16 and a keyboard 20. The computer 22 typically includesan operating system, such as Microsoft Windows or MAC OS, that supportsthe operation of other application programs, such as word processors,spreadsheets, and computer games.

The foregoing and other objectives, features, and advantages of theinvention may be more readily understood upon consideration of thefollowing detailed description of the invention, taken in conjunctionwith the accompanying drawings.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 illustrates a computer with a two-dimensional display.

FIG. 2 illustrates a computer providing a two-dimensional video signalto a two-dimensional display.

FIG. 3 illustrates a computer providing a two-dimensional video signalto a virtual reality headset with the same two-dimensional imagepresented on a respective display for each eye.

FIG. 4 illustrates a computer providing a three-dimensional video signalto a virtual reality headset with different two-dimensional imagespresented on a respective display for each eye.

FIG. 5 illustrates a computer providing a two-dimensional video signalto a 2D-3D converter that provides a three-dimensional video signal to avirtual reality headset.

FIG. 6 illustrates a virtual reality headset that may include sensor(s),speakers(s), display(s), and/or eye tracking sensor(s).

FIG. 7 illustrates a computer providing a two-dimensional video signalto a 2D-3D converter that provides a three-dimensional video signal to avirtual reality headset that provides sensor data to the 2D-3D converterthat provides the sensor data to the computer or otherwise providessensor data directly to the computer.

FIG. 8 illustrates a computer providing a two-dimensional video signalto a 2D-3D converter that provides a three-dimensional video signal to avirtual reality headset, together with selecting control vector binding.

FIG. 9 illustrates control vector binding scaling.

FIG. 10 illustrates a headset calibration technique.

FIG. 11 illustrates selection of a calibration data sets.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENT

Referring to FIG. 2, the computer, often in the environment of computergames, provides a two-dimensional video signal having HD, 4K UHD, 5KUHD, and even sometimes 8K UHD resolution, to the display. Theinterconnection between the computer and the display, depending on theparticular technology has limited bandwidth. By way of example, somecomputers are only capable of transmitting and/or some displays are onlycapable of receiving HD or less. By way of example, some computers areonly capable of transmitting and/or some displays are only capable ofreceiving 4K UHD or less. By way of example, some computers are onlycapable of transmitting and/or some displays are only capable ofreceiving 5K UHD or less. By way of example, some computers are onlycapable of transmitting and/or some displays are only capable ofreceiving 8K UHD or less. In addition, often the particular type ofinterconnection technology, DisplayPort cables, HDMI cables, DVI cables,USB-C cables, etc., limit the bandwidth of video that may be transmittedbetween the computer and the display.

Referring to FIG. 3, the computer, often in the environment of computergames, provides a two-dimensional video signal having 2.5K which is2560×1440, 4K UHD, and over 4K UHD for each eye, to a virtual realityheadset. The interconnection between the computer and the virtualreality headset, depending on the particular technology has limitedbandwidth. By way of example, some computers are only capable oftransmitting and/or some virtual reality headsets are only capable ofreceiving 2K UHD or less. By way of example, some computers are onlycapable of transmitting and/or some virtual reality headsets are onlycapable of receiving 4K UHD or less. By way of example, some computersare only capable of transmitting and/or some virtual reality headsetsare only capable of receiving 5K UHD or less. By way of example, somecomputers are only capable of transmitting and/or some virtual realityheadsets are only capable of receiving 8K UHD or less. In addition,often the particular type of interconnection technology, DisplayPortcables, HDMI cables, DVI cables, USB-C cables, etc., limit the bandwidthof video that may be transmitted between the computer and the virtualreality headset. The content provided by the computer is two-dimensionalvirtual reality content or two-dimensional 360 virtual reality content,and the content rendered on the virtual reality headset is likewisetwo-dimensional content.

Two-dimensional video content, such as obtained with a video camerahaving a single aperture, is often either projected onto a displayscreen for viewing or viewed on a display designed for presentingtwo-dimensional content. Over time, the resolution of displays hastended to increase, from standard television interlaced contentresolution (e.g., 48i), to high definition television content (e.g.,1080i), to 4K definition television content (4K UHD), and even to evenhigher definition television content (e.g., 8K UHD). Such increases invideo resolution technology only provide for limited increases in theapparent image quality to the viewer. Accordingly, the viewer is onlyimmersed in the video experience to a limited extent.

To increase the immersive experience of the viewer it is often desirableto view three-dimensional image content. The perception ofthree-dimensional content may involve a third dimension of depth, whichmay be perceived in a form of binocular disparity by the human visualsystem. Since the left and the right eyes of the viewer are at differentpositions, each perceives a slightly different view of a field of view.The human brain may then reconstruct the depth information from thesedifferent views to perceive a three-dimensional view. To emulate thisphenomenon, a three-dimensional display may display two or more slightlydifferent images of each scene in a manner that presents each of theviews to a different eye of the viewer. A variety of different displaytechnologies may be used, such as for example, anaglyphthree-dimensional system, polarized three-dimensional system, activeshutter three-dimensional system, head mounted stereoscopic displaysystem, and auto stereoscopic display system.

Referring to FIG. 4, the computer, often in the environment of computerthree-dimensional games, provides a three-dimensional video signalhaving 880×1600 (1400×1600 per eye) or 2160×1200 (1080×1200 per eye), toa virtual reality headset. The interconnection between the computer andthe virtual reality headset, depending on the particular technology haslimited bandwidth. While the computer may have relatively highthree-dimensional video content, due to rendering limitations thevirtual reality headset only receives a portion of the overall availablevideo content, such as ⅛^(th) of the three-dimensional view. By way ofexample, a virtual reality headset typically has about a 90 degree fieldof view in a horizontal and vertical direction. By way of example, the⅛^(th) may be viewed when the virtual reality is two-dimensional, with ¼of the 360 degrees horizontally and ½ of the 180 degrees vertically. Byway of example, 1/16^(th) may be viewed when the virtual reality isthree-dimensional because the frames are split in half vertically forthe left eye and the right eye. In this manner, the user of the virtualreality headset needs to adjust their view substantially in order toview the entire 4K UHD view, a portion at a time, that would haveotherwise been available on a traditional display. By way of example,rather than rendering a game as 4K UHD or 5K UHD that a conventionalcomputer game renders as two-dimensional on a two-dimensional monitor,the virtual reality headset generally displays as approximately standarddefinition equivalent number of pixels, or less (e.g., generally ⅛^(th)or 1/16^(th) of 4K UHD). Moreover, the rendered three-dimensionalcontent may be of somewhat insufficient quality due to the renderinglimitations and the three-dimensional content being provided to thevirtual reality headset.

As three-dimensional display systems become more readily prevalent thedesire for suitable three-dimensional content to present on suchdisplays increases. One way to generate three-dimensional content isusing three-dimensional computer-generated graphics. While such contentis suitable for being displayed, the amount of desirable suchthree-dimensional computer-generated content is limited. Another way togenerate the dimensional content is using three-dimensional video camerasystems. Likewise, while such video camera content is suitable for beingdisplayed, the amount of desirable such three-dimensional content islikewise limited. A preferable technique to generate three-dimensionalcontent is using the vast amounts of available two-dimensional content,especially the decades of single-view high-resolution two-dimensionalcontent available from computer games, and converting thetwo-dimensional content into three-dimensional content.

One technique to achieve single-view two-dimensional (2D) tothree-dimensional (3D) conversion is using a modified time differencetechnique. The modified time difference technique converts 2D images to3D images by selecting images that would be a stereo-pair according tothe detected motions of objects in the input sequential images. Thistechnique may, if desired, be based upon motion vector informationavailable in the video or otherwise determined.

Another technique to achieve two-dimensional (2D) to three-dimensional(3D) conversion is using a computed image depth technique. The 3D imagesare generated based upon the depth inferred characteristics of each 2Dimage. The characteristics of the image that may be used, include forexample, the contrast of different regions of the image, the sharpnessof different regions of the image, and the chrominance of differentregions of the image. The sharpness, contrast, and chrominance values ofeach area of the input image may be determined. The sharpness relates tothe high frequency content of the luminance signal of the input image.The contrast relates to a medium frequency content of the luminancesignal of the input image. The chrominance relates the hue and the tonecontent of the color signal of the input image. Adjacent areas that haveclose color may be grouped together according to their chrominancevalues. The image depth may be computed using these characteristicsand/or other characteristics, as desired. For example, generally nearpositioned objects have higher sharpness and higher contrast than farpositioned objects and the background image. Thus, the sharpness andcontrast may be inversely proportional to the distance. These values maylikewise be weighted based upon their spatial location within the image.Other techniques may likewise be used to achieve a 2D to 3D conversionof an input image, including motion compensation, if desired.

Referring to FIG. 5, the computer, often in the environment ofconventional computer games, often provides a two-dimensional videosignal having HD or 2.5K to a rendering device Accordingly, from theviewpoint of the computer, the output has a single view two-dimensionalvideo signal. Preferably, the video signal has a 4K UHD, 2.5K, 5K UHD,8K UHD, or more, resolution that is especially suitable for ahigh-resolution rendering. The two-dimensional video signal ispreferably carried by a transmission cable, such as a HDMI cable, a DVIcable, a DisplayPort cable, or a USB-C cable. The transmission cable isinterconnected to a two-dimensional to three-dimensional virtual realityvideo converter. The output of the two-dimensional to three-dimensionalvideo converter is a three-dimensional virtual reality video signal.Accordingly, from the viewpoint of the two-dimensional tothree-dimensional video converter, the output has a three-dimensionalvirtual reality video signal. Preferably, the video signal has a HD,2.5K, 4K UHD, or more, resolution that is especially suitable for ahigh-resolution three-dimensional rendering. The three-dimensional videosignal is provided to a three-dimensional enabled virtual realityheadset suitable to be worn by the head of a user. Accordingly, from theviewpoint of virtual reality headset, the input has a three-dimensionalvideo signal that was provided from the computer system. In this manner,the computer provides a two-dimensional video signal that wouldtraditionally be provided to a two-dimensional display, typically usinga high-resolution video format. In this manner, the virtual realityheadset receives a three-dimensional video signal, typically as ahigh-resolution video format. In this manner, the two-dimensional videosignal is rendered as a three-dimensional virtual reality video signal,without either the computer or the virtual reality headset being awareof the other's two-dimensional or three-dimensional video capabilities.Thus, the computer and converter can output a much higher resolutionvirtual reality three-dimensional video signal than possible with aframe based virtual reality video signal. The virtual reality headsetcan receive a high-resolution three-dimensional video input, which isnot typical for high end virtual reality video games which at mostprovide relatively low-resolution three-dimensional video, such as1/16^(th) of the pixels in a 4K virtual reality frame.

As it may be observed, there is not typically bandwidth limitations inthe cabling, but rather, that conventional virtual reality is limited toonly being able to display that portion of a 360 4K frame that is infront of the viewer. This limitation is a result of conventional virtualreality that includes all of the 360 video around the viewer in one 4Kframe. In contrast, a conventional computer or a game console onlyrenders that which is in front of the viewer, which is often at 4K orgreater resolution. While the computer may have relatively highthree-dimensional video content, due to rendering limitations thevirtual reality headset only displays a portion of the overall available4K frame based virtual reality video content, such as ⅛^(th) or1/16^(th) of the three-dimensional frame. In this manner, the user ofthe virtual reality headset when directly receiving three-dimensionalimage content from a computing device needs to adjust their viewsubstantially in order to view the entire 4K UHD view, a portion at atime, that would have otherwise been available when receiving similartwo-dimensional image content on a traditional two-dimensional display.By way of example, rather than rendering a game as 4K UHD or 5K UHD, thevirtual reality headset generally displays as standard definitionequivalent number of pixels, or less (e.g., generally ⅛^(th) or1/16^(th) of 4K UHD). Moreover, the rendered three-dimensional contentmay be of somewhat insufficient quality due to the rendering limitationsand the three-dimensional content being provided to the virtual realityheadset. By way of example, for a 4K UHD linear two-dimensional outputfrom the computer, after conversion from 2D to 3D, the input to thevirtual reality headset may be a pair of 4K images, thus being generallyequivalent to receiving 8K UHD content. Furthermore, preferably the viewillustrated in the two-dimensional image output is the same orsubstantially the same (within 10 percent) in the view of thethree-dimensional image being rendered in the virtual reality headset.In this manner, the typical wide expansive view that would have beenrendered on a display, such as a desktop liquid crystal monitor, issubstantially the same view that is rendered on the virtual realityheadset which increases the immersive experience of the viewer.

Typically, computer applications that are designed to output atwo-dimensional video output to a display include a user interface thatis based upon the user interfacing with an interface device, such as acomputer keyboard and/or a game controller and/or a mouse. By way ofexample, the mouse is moved to the right the view is moved to the right,the mouse is moved to the left the view is moved to the left, the mouseis moved up the view is moved up, and the mouse is moved down the viewis moved down. In this manner, the viewer can control the view observedon the display using the interface device. However, with a virtualreality headset covering the eyes of the user it is problematic for theuser to effectively use the interface device to control the viewobserved on the display.

Referring to FIG. 6, the virtual reality headset may include one or moresensors for tracking motion such as gyroscope sensors, accelerometersensors, may include one or more speakers for mono or stereo sound, mayprovide a separate image for each eye with one or more displays, and mayinclude one or more eye tracking sensors. The three-dimensional inputvideo provides data to display the three-dimensional content on thedisplay(s) and provide audio to the speaker(s) for the virtual realityheadset. Movement of the virtual reality headset, as sensed by thesensor(s) (which may be the eye tracking sensor(s)) can provide outputsignals indicating the movement of the virtual reality headset.

Referring to FIG. 7, the virtual reality headset may providesensor-based data, through the two-dimensional to three-dimensionalconverter, to the computer regarding its movement. The virtual realityheadset may provide sensor-based data directly (or otherwise) to thecomputer regarding its movement, such as by a USB connection or othercable. The computer may include a human interface device (e.g., driversand/or software) that receives the input from the virtual realityheadset in such a manner that it appears to be a standard controllingdevice, such as a mouse, a keyboard, or a game controller. For example,the data may be provided over any suitable transport, such as HID overI2C or HID over USB. In this manner, when the headset turns to the rightthe computer will receive data indicating the viewer turned to the rightand modify the output view accordingly. In this manner, when the headsetturns to the left the computer will receive data indicating the viewerturned to the left and modify the output view accordingly. In thismanner, when the headset turns upward the computer will receive dataindicating the viewer turned upward and modify the output viewaccordingly. In this manner, when the headset turns downward thecomputer will receive data indicating the viewer turned downward andmodify the output view accordingly. Thus, the virtual reality headset isbound to the computer for communications. Accordingly, the user canmodify the view that is observed simply by moving the headset (or eyegaze for eye tracking), which is suitable when the eyes of the viewercan not readily observe a mouse, keyboard, or game controller.

In another embodiment, the sensor data does not pass through thetwo-dimensional to three-dimensional converter. The sensor data may beinterconnected from the two-dimensional to three-dimensional converterto the computer using a different cable, or otherwise the same cable(s)with the sensor data is routed such that it doesn't pass through theconversion aspects of the two-dimensional to three-dimensionalconverter. Preferably, the headset motion controls (e.g., sensor data)are converted to HID commands (e.g., data) by software running on thecomputer.

In another embodiment, the computer may be a computer gaming console,such as an XBOX 360 or a PlayStation. In another embodiment, thecomputer may be a mobile phone that includes a three-dimensionaldisplay, with the two-dimensional to three-dimensional conversionoccurring within the mobile phone, where the mobile phone is part of avirtual reality headset and in particular provides the display for theheadset. In this embodiment, the mobile device would also include theapplication that plays a standalone and/or network-based game.

As it may be observed, the computer provides a single viewtwo-dimensional video signal that would traditionally be provided to atwo-dimensional display monitor, typically using a high-resolution videoformat, while the virtual reality headset receives a three-dimensionalvideo signal, typically as a high-resolution video format. Further as itmay be observed, the virtual reality headset, or otherwise otherhardware and/or software associated with the virtual reality headset,provides responsive data to the computer in a manner that the computerwould otherwise expect, such as a mouse, a keyboard, or a gamecontroller, to control the movement of the images displayed on thevirtual reality headset. Moreover, the computer is unaware that thecontent is being viewed on a virtual reality headset, as thetwo-dimensional to three-dimensional converter appears to the computeras an anticipated two-dimensional rendering display, and the dataprovided from the virtual reality headset appears as an anticipatedinterface device.

In this manner, the two-dimensional video signal is rendered as athree-dimensional video signal, without either the computer or thevirtual reality headset being aware of the other's two-dimensional orthree-dimensional video capabilities. Thus, the computer can output ahigh-resolution two-dimensional video signal, which is typical for highend video games, and the virtual reality headset can receive ahigh-resolution three-dimensional video input, which is not typical forhigh end video games which at most provide relatively low-resolutionthree-dimensional video. Moreover as it may be observed, that video datafrom the computer is two-dimensional video content which is relativelylower bandwidth compared to the corresponding three-dimensional videocontent which is relatively higher bandwidth or the same bandwidth, andaccordingly the computer only need the capability of providing therelatively lower bandwidth two-dimensional video. Moreover, as it may beobserved, there is no need to cache a plurality of frames ofthree-dimensional video content for the virtual reality headset becauseit is converted ‘on the fly’ from the two-dimensional video content.

Referring to FIG. 8, the binding of the virtual reality headset may beperformed in a manner consistent with the particular application it isbeing used for. The different vectors, such as for example, position,motion, turning left and right, turning upward and downward,acceleration, etc., from the virtual reality headset, or otherwise otherhardware and/or software associated with the virtual reality headset,may be mapped (e.g., bindings) in a different manner by the computerdepending on the context. By way of example, for a medical applicationthe virtual reality headset may include a first set of bindings whichmap vectors from the virtual reality headset to a traditional interfacedevice in a first manner. By way of example, for a first-person shootergame the virtual reality headset may include a second set of bindingswhich map vectors from the virtual reality headset to a traditionalinterface device in a second manner, different from the first manner.For example, the first manner may be consistent with a computer mouse.For example, the second manner may be consistent with a game controller.For example, a third manner may be consistent with particular keys of akeyboard. The particular binding may be selected automatically in somemanner based upon the particular application being used. The particularbinding may be selected on the virtual reality headset in some manner,such as setting made through the display on the virtual reality headset.In this manner, the user may select the desired manner of bindingwithout regard to the computer. The particular binding may be selectedon the computer in some manner, such as setting made through thecomputer for the particular application being used. In this manner, theuser may select the desired manner of binding without regard to thevirtual reality headset.

Referring to FIG. 9, the manner of binding may further include one ormore different scaling factors, which may be selectable by the user. Thescale factor may be program specific, if desired. The scale factor mayalso include a maximum and a minimum, if desired, which may be furtherprogram specific, if desired. The user may enter the scaling for each ofthe movements of the virtual reality headset. In some cases, the usermay prefer a small movement of the head resulting in a substantialmovement of the view being observed. In other cases, the user may prefera large movement of the head resulting in an insubstantial movement ofthe view being observed. In many cases, the scaling may be used tocompensate for different games (or other software) having differentsensitivities (e.g., mouse, joystick, keys, and/or trackball). Forexample, “turn right” may be as a result of moving the mouse right,“turn left” may be as a result of moving the mouse left, “look upward”may be as a result of mouse up, “look downward” may be as a result ofmouse down, etc. For example, a set of keys may be used for the motioncontrols, such as A, B, C, D, →, ↓, ↑, ←.

Referring to FIG. 10, another manner of binding may include aninteractive technique. The user may initiate calibration of the virtualreality headset. The user may make a first movement, such as turn left90 degrees as indicated on the display or audibly, and then indicate themovement has been made, such as by clicking a mouse. The user may make asecond movement, such as turn right 90 degrees as indicated on thedisplay or audibly, and then indicate the movement has been made, suchas by clicking a mouse. The user may make a third movement, such as turnupward 90 degrees as indicated on the display or audibly, and thenindicate the movement has been made, such as by clicking a mouse. Theuser may make a fourth movement, such as turn downward 90 degrees asindicated on the display or audibly, and then indicate the movement hasbeen made, such as by clicking a mouse. Other calibrations may likewisebe made. The result is a set of calibration data suitable for being usedparticularized to the user and/or the particular application. A similartechnique may involve the use of one or more key presses to achieve acalibration.

Referring to FIG. 11, when a user is going to use the virtual realityheadset, the user may select a particular application that the virtualreality headset is going to be used with and thereby select acalibration set particular to that application. In addition, each of thecalibration sets may be customized for each particular user. In thismanner, a different calibration set may be used for differentapplications. Also, in this manner, a different calibration set may beselected for the same application for different users.

In some embodiments, the headset may include a set of one or moreexternal sensors affixed thereto, such as gyroscopic sensors, and theexternal sensors are used for calibration. Such calibration may include,for example, yaw, pitch, roll, and acceleration.

In some embodiments, the headset may be calibrated based upon otherangular movements, such as 45 degrees, where movement of 45 degrees istranslated into a movement of 90 degrees.

In a typical virtual reality game, the computer renders all the graphicsfor a 360 degree view, including those views behind the view of theuser, where the headset only displays a small portion of the view at anyparticular time. As the head of the viewer turns it renders additionalportions of the 360 degree view, which reduces the latency when theuser's head turns. In contrast, the computer for a computer game itrenders only that portion that is intended to be viewed on the displayat any particular point in time. Since the computer for a computer game(or otherwise) only renders that portion that is intended to be viewedat any particular point in time, the visible resolution is suitable tobe much greater than the rendering associated with a traditional 360degree virtual reality.

In a typical embodiment, the video frame that the headset receives is ina polar coordinate system suitable to wrap a 360 degree space into arectangular frame. In this manner, the headset needs to convert its viewthat is going to be rendered to normal rectangular coordinates for theheadset screen, requiring significant computational resources. However,when the headset receives video content that is to be rendered which isalready in normal rectangular coordinates, such as three-dimensionalvideo from the converter, the computational requirements are simplifiedsince the polar conversion is no longer necessary.

As it may be observed, when general typical video is captured andprovided in a series of linear frames of reference, such as in X-Ycoordinates, which is what is viewed on a monitor or movie screen. Thecomputer then converts the linear frames of reference to a differentcoordinate system, such as a polar coordinate system, suitable to wrap a360 degree space into a rectangular frame. Then a fraction of the polarcoordinate space based frame of video content is then provided fordisplay on a virtual reality headset display, such as 90 degreesvertical and 90 degrees horizontal, depending on the view that isselected. The 90 degrees vertical and 90 degrees horizontal may beextracted from the linear view, which tends to have relatively lowerdistortion, or may be computed based upon the polar view frame, whichtends to have relatively higher distortion. The 90 degrees vertical and90 degrees horizontal is generally 1/8th of a frame that is observed.With the three-dimensional display, half of the ⅛^(th) is provided forthe left eye and half of the ⅛^(th) is provided for the right eye,resulting in rendering 1/16^(th) for each eye. In this manner, theresolution provided to each eye is generally about standard definitionwhich is relatively low-resolution. Unfortunately, for a virtual realitygame it is necessary to render the entire 360 degree space for eachframe, which is computationally expensive and slows down the speed atwhich the virtual reality game renders, which could inhibit the user'sexperience.

In contrast, it is more desirable to receive the general typical videothat is captured and provided in a series of linear frames of reference,such as in X-Y coordinates, and omit the conversion of the series oflinear frames of reference to another coordinate system, such as thepolar coordinate system. The series of linear frames of reference may beconverted to 3D and directly provided to the virtual reality headset,either at full resolution or at a reduced resolution. The virtualreality headset may receive the series of linear frames of reference andmay include internal processing to convert the two-dimensional framesinto three-dimensional frames. A control signal may be provided from theuser or otherwise the virtual reality headset or otherwise a computingdevice associated with the virtual reality headset, to indicate to thecomputing device providing the series of linear frames of referencewhich frames of reference to provide. The control signal is preferablyprovided in the form of a human interface device (HID) based signal. Inthis manner, it is not necessary to render the entire 360 degree spacefor each frame, which is computationally expensive and slows down thespeed at which the virtual reality game renders, which could inhibit theuser's experience, and thereby improve the user's experience and reducethe computational expense of the system. Thus, this enableshigh-resolution virtual reality pixel detail that was not otherwisereadily obtainable.

The embodiments described facilitate the use of the vast library ofexisting two-dimensional content and gaming content to be used in anexpanded manner. Further, the full resolution of the series of linearframes may be used for the conversion from two-dimensional image contentto three-dimensional image content. Also, by using the series of lineartwo-dimensional frames, the converted two-dimensional image content tothree-dimensional image content tends to have an improvedthree-dimensional visual appearance, with up to generally 32 times morepixel detail with 4K UHD virtual reality headsets (depending on therelative resolutions) than the typical three-dimensional output from asoftware program operating on the computing device to a virtual realityheadset. Moreover, by making direct use of the two-dimensional content,and the tools used in creating games based upon two-dimensional content,three-dimensional content may be derived in a simplified manner whichextends the user experience of the game.

The terms and expressions which have been employed in the foregoingspecification are used therein as terms of description and not oflimitation, and there is no intention, in the use of such terms andexpressions, of excluding equivalents of the features shown anddescribed or portions thereof, it being recognized that the scope of theinvention is defined and limited only by the claims which follow.

I/We claim:
 1. A method for conversion of a series of two-dimensionalimages into a series of three-dimensional images comprising: (a)receiving said two-dimensional images from a video source at atwo-dimensional to three-dimensional converter where each of said seriesof two-dimensional images are from a same viewpoint, where said seriesof two-dimensional images do not include any images from a viewpointother than said same viewpoint, where said two-dimensional images arereceived in a format suitable for being displayed on a two-dimensionaldisplay that is not capable of displaying three-dimensional video, wheresaid two-dimensional images are received at a frame rate consistent withbeing displayed on said two-dimensional display; (b) saidtwo-dimensional to three-dimensional converter processing saidtwo-dimensional images all of which are from said same viewpoint and donot include any images from a viewpoint other than said same viewpointto determine said three-dimensional images; (c) said two-dimensional tothree-dimensional converter providing said three-dimensional images in aformat suitable for being displayed on a three-dimensional display,where said three-dimensional images are provided at a frame rateconsistent with being displayed on said three-dimensional display; (d)receiving said three-dimensional images by said three-dimensionaldisplay and in response rendering said three-dimensional images on saidthree-dimensional display.
 2. The method of claim 1 wherein said wheresaid two-dimensional images are received by said two-dimensional tothree-dimensional converter at a first bandwidth rate and wherein saidthree-dimensional images are provided by said two-dimensional tothree-dimensional converter at a second bandwidth rate, wherein saidsecond bandwidth rate at least one of is greater than and equal to saidfirst bandwidth rate.
 3. The method of claim 1 wherein said where saidtwo-dimensional images are received by said two-dimensional tothree-dimensional converter at a first bandwidth rate and wherein saidthree-dimensional images are provided by said two-dimensional tothree-dimensional converter at a second bandwidth rate, wherein saidsecond bandwidth rate is the same as said first bandwidth rate.
 4. Themethod of claim 1 wherein where said video source is not aware that saidtwo-dimensional images are to be displayed on said three-dimensionaldisplay.
 5. The method of claim 4 wherein where said three-dimensionaldisplay is not aware that said three-dimensional images originated assaid two-dimensional images.
 6. The method of claim 1 wherein saidtwo-dimensional images are suitable to rendered with a maximum firstview and said three-dimensional images are suitable to be rendered witha maximum second view that are different based upon pixel displacements.7. The method of claim 1 wherein said three-dimensional display is athree-dimensional enabled virtual reality headset.
 8. The method ofclaim 7 wherein said three-dimensional display includes at least onesensor for tracking motion of said three-dimensional display.
 9. Themethod of claim 8 wherein said at least one sensor includes at least oneof a gyroscope, an accelerometer, and a position tracker.
 10. The methodof claim 8 wherein said three-dimensional display provides an motionsignal that is received by said video source.
 11. The method of claim 10wherein said video source modifies said two-dimensional video based uponreceiving said motion signal.
 12. The method of claim 11 wherein saidmotion signal is provided to said video source through saidtwo-dimensional to three-dimensional converter.
 13. The method of claim11 wherein said motion signal is at least one of received by andconverted to said video source as a human interface device.
 14. Themethod of claim 13 wherein said human interface device is bound in amanner based upon a particular context being used.
 15. The method ofclaim 14 wherein said context is a computer mouse.
 16. The method ofclaim 14 wherein said context is a keyboard.
 17. The method of claim 14wherein said binding includes scaling factors.
 18. The method of claim14 wherein said binding is based upon an interactive process.
 19. Themethod of claim 14 wherein said binding is based upon a selection of oneof a plurality of calibration sets.
 20. The method of claim 14 whereinsaid context is a game controller.
 21. The method of claim 17 furthercomprising selectively modifying said scaling factors.